IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES A statistical–mechanical view on source coding: physical compression and data compression
نویسنده
چکیده
We draw a certain analogy between the classical information–theoretic problem of lossy data compression (source coding) of memoryless information sources and the statistical mechanical behavior of a certain model of a chain of connected particles (e.g., a polymer) that is subjected to a contracting force. The free energy difference pertaining to such a contraction turns out to be proportional to the rate– distortion function in the analogous data compression model, and the contracting force is proportional to the derivative this function. Beyond the fact that this analogy may be interesting on its own right, it may provide a physical perspective on the behavior of optimum schemes for lossy data compression (and perhaps also, an information– theoretic perspective on certain physical system models). Moreover, it triggers the derivation of lossy compression performance for systems with memory, using analysis tools and insights from statistical mechanics. A statistical–mechanical view on source coding 2
منابع مشابه
IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Data Processing Bounds for Scalar Lossy Source Codes with Side Information at the Decoder
In this paper, we introduce new lower bounds on the distortion of scalar fixedrate codes for lossy compression with side information available at the receiver. These bounds are derived by presenting the relevant random variables as a Markov chain and applying generalized data processing inequalities a la Ziv and Zakai. We show that by replacing the logarithmic function with other functions, in ...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Subset-sum phase transitions and data compression
We propose a rigorous analysis approach for the subset sum problem in the context of lossless data compression, where the phase transition of the subset sum problem is directly related to the passage between ambiguous and non–ambiguous decompression, for a compression scheme that is based on specifying the sequence composition. The proposed analysis lends itself to straightforward extensions in...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES On the Statistical Physics of Directed Polymers in a Random Medium and Their Relation to Tree-Structed Lossy Compression
Using well–known results from statistical physics, concerning the almost–sure behavior of the free energy of directed polymers in a random medium, we prove that a certain ensemble of tree–structured rate–distortion codes with delayless decoding, asymptotically achieves the rate–distortion function under a certain symmetry condition.
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Universal Quantization for Separate Encodings and Joint Decoding of Correlated Sources
We consider the multi-user lossy source-coding problem for continuous alphabet sources. In a previous work, Ziv proposed a single-user universal coding scheme which uses uniform quantization with dither, followed by a lossless source encoder (entropy coder). In this paper, we generalize Ziv’s scheme to the multi-user setting. For this generalized universal scheme, upper bounds are derived on th...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Zero-Delay and Causal Single- User and Multi-User Lossy Source Coding with Decoder Side Information
We consider zero-delay single-user and multi-user lossy source coding with decoder side information. The zero-delay constraint translates into causal (sequential) encoder and decoder pairs as well as the use of instantaneous codes. For the single-user setting, we show that optimal performance is attained by time sharing at most two scalar encoder-decoder pairs, that use zero-error side informat...
متن کامل